Stochastic Variational Inference for Gaussian Process Latent Variable Models using Back Constraints

Gaussian process latent variable models (GPLVMs) are a probabilistic approach to modelling data that employs Gaussian process mapping from latent variables to observations. This paper revisits a recently proposed variational inference technique for GPLVMs and methodologically analyses the optimality and different parameterisations of the variational approximation. We investigate a structured variational distribution, that maintains information about the dependencies between hidden dimensions, and propose a mini-batch based stochastic training procedure, enabling more scalable training algorithm. This is achieved by using variational recognition models (also known as back constraints) to parameterise the variational approximation. We demonstrate the validity of our approach on a set of unsupervised learning tasks for texture images and handwritten digits.